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1.  INTRODUCTION 

This  paper  is  concerned  with  the  statistical  assessment  of  dependency 
beyond  autocorrelation  in  the  context  of  nonlinear  time  series  models.  The 
central  theme  is  that  residuals  from  models  fitted  according  to 
inapplicable  linearity  assumptions  can  also  profitably  be  used  for  further 
analysis  beyond  linearity.  A  well  known  property  of  linear  time  3eries 
residuals  is  that  they  are  uncorrelated;  in  the  context  of  linear  models 
they  should  also  be  independent,  apart  from  the  effects  of  parameter 
estimation  with  short  series.  However,  in  the  context  of  nonlinear  models 
it  is  not  often  recognised  that  uncorrelated  residuals  also  hold 
information  concerning  higher  order  dependence  in  the  data.  Developments 
of  such  a  higher  order  residual  analysis  will  be  explored  here,  in 
particular  for  two  types  of  nonlinear  autoregressive  model  which  have  the 
usual  linear  Yule-Walker  autoregressive  correlation  structure.  Some  higher 
order  dependency  correlations  will  be  obtained.  In  a  later  paper,  the 
suggestion  of  reversed  residuals  will  be  made,  and  the  analysis  given  in 
this  paper  will  be  extended  to  encompass  these  reversed  residuals. 

2.  AOTOREGRESSION  AND  LINEAR  AUTOREGRESSIVE  RESIDUALS 
2.1  Autoregression  for  Nonlinear  Models 

The  standard  form  of  autoregression  needs  widening  for  use  with 
nonlinear  models;  we  consider  first  the  standard  form  which  is  explicitly 
autoregressive  in  a  linear  additive  way  and  then  several  weaker  variants. 
A  stationary  time  series  (Xt)  of  mean  a   is  assumed. 

Under  the  linear  autoregressive  model,  of  order  p,  the  {X^}  satisfy  the 
equation 

Xt-M=a-l(Xt_1-ji)+a2(Xt_2-M)+.  .  .  +ap(  Xt_p-M  )+«t '    t-0,±l,±2 (2.1) 

where  the  «<-  are  independent  and  identically  distributed  and  y.,  a1(  a2 ,  .  .  .  ,  orp 
are  fixed  parameters;  these  parameters  are  also  assumed  to  satisfy  the 
condition  that  the  polynomial  l-c^z-.  .  .  -<JpzP  has  all  roots  outside  the  unit 
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circle. 

A  more  general  definition  of  autoregression  of  order  p,  could  be  the 
linear   conditional   expectation   requirement  that 
E(Xt-MlXt_1,Xt_2,  .  .  .  ) 

-  a1(Xt_1-M)+a2<xt-2-*A)+-  •  • +ap(  Xt_p-^  ) ,    t«0,±l,±2 (2.2) 

Note  that  the  conditional  expectation  is  with  respect  to  all  previous  Xt . 
The  definition  (2.1)  implies  (2.2)  but  not  vice-versa.  Thus  this 
definition  could  apply  to  models  which  are  not  of  the  linear  form  (2.1), 
either  because  the  «t  are  dependent,  but  still  with  E(  <st  IXt-i,Xt-2'  •  •  •  )~°- 
or  because  the  model  has  some  other  structure  altogether.  For  instance, 
there  are  the  random  coefficient  models  of  Nicholls  and  Quinn  (1982),  the 
exponential  distribution  random  coefficient  models  of  Lawrance  and  Lewis 
(1981,1985),  the  discrete  distribution  random  coefficient  models  of  Jacobs 
and  Lewis  (1983),  and  the  gamma-beta  random  coefficient  models  of  Lewis 
(1981  ). 

Random  coefficient  autoregressive  models  of  order  p  take  the  general 

form 

*t  "  At( 1)    Xt_1+At(2)xt_2+. . .+At(P)Xt_p+Bt,     t=0,±l,±2, . . .    (2.3) 

where  the  vector  of  coefficients  {A^  1  K  A-^  ^  )/  #  ,  t  ,A^^  &K  B^ )  is  a  stationary 

vector   sequence   of   independent   random   variables,    independent   of 

X-t-i , X-fc-2 >  •  •  •  «  and  sometimes  in  addition,  B^  is  independent  of  the  random 

coefficients  A^  x )  at  time  t.    A  roots  condition,  analogous  to  that 

required  for  (2.1)  is  also  needed.   It  is  easy  to  3ee  that  such  models 

satisfy  the   linear   conditional   expectation   definition   of  pth   order 

autoregression,  but  are  nonlinear  in  the  3ense  of  (2.1);  see  also  Lawrance 

and  Lewis  (1985)  for  further  discussion  of  the  meaning  of  nonlinearity .   A 

first  order  case  of  the  type  (2.3)  will  be  u3ed  to  illustrate  the  proposed 

method  of  residual  analysis  of  autoregressive  nonlinearity. 

A  further  and  weaker  definition  of  autoregression  is  the  requirement 

that  the  autocovariances  of  the   {X^.},   denoted  by   {yjcK   just  satisfy 

Yule-WaUcer  linear  difference  equations  of  the  form, 
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>'r-al'>'r-l-,-a2'>'r-2+-  •  •+apyr-p'    Yz-Y-r>        r-1,2 (2.4) 

for  suitable  constants  oti,a2,  .  .  .  ,otp;  this  will  be  referred  to  as 
Yule-Walker  autor egression.  It  is  true  for  processes  which  satisfy  (2.1) 
and  (2.2),  as  may  be  verified  in  the  usual  manner  by  multiplying  X^  by  X^-r 
and  taking  expectations.  The  reverse  is  not  true;  (2.4)  does  not  Imply 
(2.1)  or  (2.2).  A  case  in  which  (2.4)  holds  but  in  which  (2.1)  and  (2.2) 
do  not,  is  the  product  autoregression  model  of  McKenzie  (1982),  where,  with 
p~l,  E(Xt  |Xt-i,  Xt-2<  ...  )  is  a  fractional  power  of  X^-i;  this  model  will 
also  be  used  in  Section  4.3  as  an  ilustration  of  the  proposed  residual 
methods . 

A  variety  of  other  models  can  satisfy  the  Yule-Walker  definition  of 
autoregression,  and  yet  not  satisfy  the  linear  conditional  expectation 
definition.  Amongst  these  are  first  order  Markov  chains  under  certain 
conditons,  and  others  such  as  the  semi-Markov  generated  processes  discussed 
in  Cox  and  Lewis  (Chapter  7,  1966)  and  Lewis  (1980).  Notice  that  we  do  not 
define  nonlinear  autoregressive  models  in  a  constructive  way.  The  class  is 
so  wide  as  to  make  this  impossible;  one  such  class  has  been  studied  by 
Jones  (1978).  Rather,  we  require  that  the  autocorrelations  should  satisfy 
linear  equations,  similar  in  structure  to  those  satisfied  by  the 
autocorrelations  of  linear  autoregressive  models.  In  view  of  this,  our 
suggested  analysis  extends,  rather  than  supercedes,  conventional  methods. 

2.2  Definition  and  Discussion  of  Linear  Autoregressive  Residuals 

For  the  analysis  of  time  series  data  involving  models  satisfying  (2.1), 
or  (2.2)  or  minimally  (2.3),  the  use  of  linear  autoregressive  residuals  of 
order  p,  defined  as 

Rt(P)  -  (Xt-M)-a1(Xt_1-*0-a2(Xt_2-n)-.  •  . -ap(  Xt-p->x  )         (2.5) 
is  suggested.   This  suggestion  is  based  on  the  following  theorem,  which  is 
a  generalization  of  a  result  given  in  Lawrance  and  Lewis  ( 1984,  Section 
7.2)  for  p-2 . 


Theorem.    Let  the  stationary  process  (X^}  satisfy  the  Yule-Walker  type 

equations  (2.4).   Then  the  linear  autoregressive  residuals  {R^P^}  defined 

at  (2.5)  are  uncorrelated  (although  not  necessarily  independent). 

Proof.   The  autocovariances  of  the  residuals  (2.5)  are 

Cov[Rt<P>,Rt+r<P>]    -  CovC(Xt-pi),Rt+r(p)]-«lCov[(Xt_1->i),Rt+r(p)]---- 

-apCov[(Xt-p-n),Rt+r(P)]  <2-6) 

-  Cov[(Xt-M),Rt+r(P)]-^lCovC(Xt-jO,Rt+r+l(P)]--  '  ' 

-apCov[(Xt-M),Rt+r+p<P)].  (2.7) 

Equation   (2.7)   follows  because   the   {X^}   process   is   stationary  and 

consequently  the  {Rt^P^J  process  is  stationary.   The  covariances  in  (2.7) 

need  only  be  considered  for  positive  lag  since  autocovariance  is  an  even 

function  of  r.   Then  the  crosscovariances  on  the  right-hand  side  of  (2.7) 

are  all  of  the  same  type  and  given  by 

Cov[(Xt-n),Rt+r<P)]  -  Cov[(xt-n),  ((xt+r-M)-«l(xt+r_1-M).  .  . -ap(  Xt+r_p-*i ) }  ] 

"  Yr-viyr-l- -  •  •  -^p^r-p      r_1  ' 2 (2.8) 

Now  by  the  Yule-Walxer  equations  (2.4),  the  expression  (2.8)  is  zero.   Thus 

using  (2.8)  in  (2.7) 

Cov[Rt(P),Rt+r(P)]  *  0       r-±l,±2 (2.9) 

as  was  to  be  proven.   The  proof  is  immediate  for  the  linear  autoregressive 

model  (2.1),  since  R-fc(P)  -  «•£,  and  e^     and  R^+r^P^  are  by  definition 

independent . 

Note  that  the  linear  autoregressive  residuals  {R^P^}  will  generally 
still  be  dependent  though  uncorrelated  in  nonlinear  modelling  of  the  {X^-}; 
with  the  linear  autoregressive  model  (2.1)  the  (R^P^)  will  not  only  be 
uncorrelated  but  also  independent.  It  is  this  difference  union  uill  be 
exploited  to  explore  nonlinearity  in  pth  order  nonlinear  autoregressive 
processes .  The  dependency  attributable  to  parameter  estimation  is  taken  to 
be  small  in  the  large  scale  applications  we  have  in  mind;  indeed,  nonlinear 
modelling  of  short  series  of  data  may  well  be  hard  to  justify. 

The  quantities  R-t^P^  are  autoregressive  residuals  in  the  sense  that 

r^P)  is  the  residual  of  X-t  after  subtracting  off  a^P\     its  best  linear 
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least  squares  predictor  in  terms  of  Xfc_i,Xt_2, . . . ,  given  by 

Mt( P )  -  M+a1(  Xt-i-M )+a2(  Xt-2"M >+•  •  • +«p(  Xt-p-M ) •  (2.10) 

Thus  the  residuals  (Rt(ph  give  the  basic  way  of  taking  out  the  linear 
correlation  component  in  models  with  pth  order  autoregression  minimally  of 
the  'Yule-Walker'  form  (2.4). 

A  further  point  worth  noting  about  the  residuals  R^(P)  concerns  their 
crosscovariances  with  the  Xt's.   In  the  proof  of  the  theorem  it  is  seen 

that  Cov[Xt-M,Rt+r(p)]  -  0  for  r  -  1,2 However,  the  other  half  of 

these  crosscovariances  is  non-zero. 

Example:   The  (  nonGaussian)  linear  AR(  1 )  model 

Taking  (2.1)  with  p-l,and  p  instead  of  a^,  the  crosscorrelation  of  XfM  and 

Rt-r^P)  in  the  AR(  1 )  model  is  given  by 

Corr(Xt-M,Rt-r)  ■  ( 1~P2  )1/2  Pr   for  r-1,2,3 ;      (2.11) 

the  superscript  has  been  dropped  from  Rt-r'  a3  it  will  be  in  similar  future 
u3e.  Note  further,  that  in  this  case  Rt+r  is  independent  of  X-t  for 
r-1,2,...  .   Further  use  of  (2.11)  will  be  made  in  Sections  3  and  5. 

For  the  use  of  {Rt^pM  in  data  analysis,  the  order  p  of  the  linear- 
aspect  of  the  autoregression  needs  to  have  been  chosen;  any  of  the 
available  standard  methods  may  still  be  used.  In  addition,  of  course,  the 
ixtai,a2,  .  .  .  ,ctp  need  to  be  estimated;  there  are  at  least  two  convenient 
possibilities:  (1)  the  assumption  of  a  linear  autoregressive  model  like 
(2.1)  and  the  use  of  least  squares  estimation,  and  (2)  a  non-model  based 
approach  to  estimation  employing  the  first  p  Yule-Walker  type  equations 
(2.4).  The  latter  is  suggested  here;  however,  Tjostheim  and  Paulsen  (1983) 
recommend  (1)  when  dealing  with  modest  sized  samples  from  linear  models,  on 
account  of  serious  estimation  bias  with  the  Yule-Walker  estimates.  Since 
the  use  envisaged  here  is  primarily  with  nonlinear  models  and  substantial 
sets  of  data,  the  superiority  of  (1)  over  (2)  is  not  established  or 
crucial. 


3.   ASSESSMENT  OP  HKiHEK  ORDER  DEPENDENCE 

Since  the  linear  autoregressive  residuals  {Rt^^}  of  (2.5)  have  zero 
covariances  when  (2.4)  holds ,  but  for  nonlinear  processes  need  not  be 
independent,  a  residual  analysis  of  this  nonlinearity  can  be  based  on  an 
assessment  of  their  higher  order  dependence.  To  consider  what  form  this 
might  take,  we  note  that  the  use  of  {X^2}  has  been  suggested  by  Granger  and 
Andersen  (1978, p. 63)  for  bilinear  models,  for  which  in  many  of  the  simple 
cases  the  Xfc  have  zero  autocorrelations  themselves.  The  corresponding 
initial  suggestion  here  for  a  residual  analysis  is,  as  was  briefly 
illustrated  in  Lawrance  and  Lewis  (1985),  to  use  the  residuals  ((Rt^M2}- 
Displays  can  then  easily  be  made  of  the  associated  autocorrelation 
functions,  scatter  plots,  periodograms ,  cumulative  periodograms ,  etc,  using 
standard  ( second  order )  time  series  software . 

However,  the  autocorrelations  of  {(Rt^M2}  axe  fourth  order  quantities 
in  the  original  series  {X^},  which  is  a  double  jump  from  the  second  order 
autocorrelations  of  the  series  (X-t).  Such  quantities  will  be  very 
difficult  to  handle  theoretically  with  most  types  of  nonlinear  model. 
The  crosscorrelation  function  of  {Rt^M  and  {fRt^^2}  is  essentially 
third  order  and  is  preferred  here;  for  previous  use  3ee  Subba  Rao  and  Gabr 
(1984)  and  Lawrance  and  Lewis  (1985).  Since  this  crosscorrelation  function 
involves  the  variance  of  {(Rt^^)2}/  it  also  needs  some  fourth  order  joint 
moments  of  {X^}  up  to  lag  p.  The  necessary  calculations  of  all  these 
quantities  for  two  nonlinear  models  will  be  given  in  Section  4. 

The   behaviour   of   the   crosscorrelation   function   of   {Rt^M   and 

{(Rt(P))2}  may  be  judged  against  the  fact  that  the  (Rt(P)}  are  independent 

for   the   linear   autoregressive   model   of   order   p,   and   hence   the 

crosscorrelation  function  of  {Rt^^}  and  ((Rt'P^)*}  will  be  zero  except  at 

lag  zero.   For  the  random  coefficient  autoregressive  processes  considered 

in  Section  4,  the  suggested  crosscorrelation  function  will  be  shown  to 

posess  a  useful  rut-off  property;  this  generalizes  the  NEAR( 2  )  result  given 
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in  Lawrance  and  Lewis  (  1985 ) . 

Two  rather  simpler  crosscorrelation  quantities  can  be  proposed  from 
consideration  of  the  zero  covariance  result  of  (2.8),  and  the  additional 
fact  that  when  the  pth  order  linear  autoregressive  model  holds,  Xt  and 
Rt+r^^  (r-1,2,...)  will  be  independent,  and  not  just  uncorrelated . 
Working  in  terms  of  {X^}  adjusted  for  its  mean  ji,  which  is  better 
computationally  and  often  nicer  theoretically,  the  following  quantities  may 
be  considered, 

Corr[(Xt-M)2,Rt+r(P)]'     forr-0,±l,±2 (3.1) 

Corr[Xt-M,(Rt+r(P))2J'     for  r-0±l, ±2, . . .  .  (3.2) 

These  autocorrelations  are  not  equivalent  in  the  aspects  of  higher  order 
dependency  of  the  (Xt)  process  which  they  assess.  To  3ee  this,  note  that 
the  covariance  corresponding  to  (3.1)  involves  only  third  order  joint 
moments  of  the  form  E[(Xt~M )2(  Xt+r-i~M ) ] <  i-0, ±1, . . . , ±p  while  the 
covariance  corresponding  to  (3.2)  involves  additional  joint  moments  of  the 
form  E[X^--m)(  Xt+r~^  K  xt+r-i  )]  >  i*0,  *1,  .  .  .  ,  *p.  Further,  the  denominator  of 
(3,2),  by  needing  var((Rt^P^  )2} ,  is  more  complicated  in  the  higher  order 
moments  it  involves  relative  to  the  denominator  of  (3.1)  which  needs 
varfRt^P)}-  However,  both  correlations,  and  particularly  (3.1),  are  more 
tractable  than  the  autocorrelations  of  ((Rt^P^)2}  or  the  crosscorrelations 
of  (Rt(p))  and  {(Rt(p))2}- 
Example  (Continued  from  (2.11)):   The  ( nonGaussian )  linear  AR( 1 )  model. 

As  an  illustration  of  the  use  of  (3.1)  and  (3.2),  for  the  linear  AR( 1 ) 

model,  ((2.1)  with  p-1  and  p  instead  of  o^ )  there  are  the  results 

TO  r— 1,-2,  .  .  . 


Corr[(Xt-M),(Rt-r>2] 


-9kew(X)  [vIr(rR^rT(1-p3)pr'     -0,1,... 


Corr[(Xt-M2,Rt-r] 


fO  r— 1,-2,... 

(3.4) 
Var(X)         1-P*  .sr 


l3kew(X)n7arT73^)Jn^  IT-^P "   -0,1,... 

Note  the  faster  geometric  decrease  in  (3.4),   for  non  negative  lags, 
relative  to  (3.3);  also  (3.3)  contains  more  higher  moment  information.   The 
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behaviour  of  these  functions  —  zero  for  r<0  and  geometric  decay  for  r£0  - 
suggests  linear  models  with  skewed  marginal  distributions  or  nonlinear 
models.  Discussion  of  parallel  results  for  two  types  of  nonlinear  model 
are  given  in  section  4. 

The  use  of  squaring  in  the  construction  of  these  higher  order 
dependency  measures  is  recognized  as  being  pragmatic  and  somewhat  arbitary; 
it  does  however  lead  to  expressions  involving  selected  types  of  simpler 
higher  order  cross  moments.  The  end  use  of  the  higher  order  dependency 
measures  can  either  be  exploratory  for  a  given  data  3et,  to  ascertain 
whether  there  is  appreciable  nonlinearity  present,  or  constructively,  to 
provide  evidence  for  fitting  suitable  types  of  nonlinear  model  which  can 
match  the  observed  higher  order  dependency. 

Earlier  discussion  of  higher  order  dependence  in  nonGaussian  linear 
models  is  given  by  Rosenblatt  (1980)  in  terms  of  the  bispectrum;  aspects  of 
nonlinearity  and  higher  order  spectra  are  briefly  considered  in  Rosenblatt 
(1979). 


4.   HIGHER  ORDER  DEPENDENCY  FOR  TWO  TYPES  OP  NONLINEAR  MODEL 

In  this  section  we  obtain  properties  of  the  measures  of  higher  order 
dependency  discussed  in  the  previous  section,  for  two  specific  types  of 
nonlinear  models.    The  models  considered   are  autoregressive   in  the 
Yule-Walker  sense  of  (2.4),  but  not  in  the  linear  sense  of  (2.1). 
4.1   Random  Coefficient  Autoregressive  Models 

As  already  remarked  in  Section  1,  a  generalization  of  the  linear 
autoregressive  model  (2.1)  is  to  let  the  coefficients  a1,a2,...ap  be  random 
variables.  One  general  class  of  such  models  has  been  discussed  by  Nicholls 
and  Quinn  (1982)  who  cite  Andell  (1976)  and  articles  in  the  economic 
literature.  Other  classes  of  models  with  random  coefficients  include  the 
discrete  distribution  models  of  Jacobs  and  Lewis  (1983)  and  the  exponential 
models  of  Lawrance  and  Lewis  (1981,1985).  The  class  of  random  coefficient 
autoregressive  processes  considered  here  is  given  by 

Xt  -  At(1)xt_1+At(2)Xt_2+.  .  .+At(PbCt_p+Bt        t«0,±l,±2 ,    (4.1) 

where  (A^1  ),A-t^  2  K  .  .  .  ,At^P^,Bt }  is  independent  of  Xt-i/Xt-2 /  •  •  •  and  forms 
a  stationary  vector  sequence  of  Independent  random  variables  where 
E(At^^)"Cfj  for  j-l,...,p  satisfy  the  standard  polynomial  equation.  The 
components  of  the  vectors  are  not  necessarily  independent.  For  example  the 
discrete  distribution  models  of  Jacobs  and  Lewis  (1983)  can  be  written  in 
this  form,  and  have  dependent  coefficients,  as  do  the  exponential  models  of 
Lawrance  and  Lewis  (1981,  1985).  It  is  easily  verified  that  the  process 
(4.1)  satisfies  conditional  expectation  autoregression  (2.2)  and  thus  also 
the  weaker  Yule-Walker  definition  (2.4);  it  will  also  clearly  be 
stationary.  Note  as  well  that  the  standard  linear  autoregressive  model 
(2.1)  is  a  special  case  of  (4.1)  in  which  the  random  coefficients  are 
actually  constant. 

We  now  give  a  characteristic  result  for  this  type  of  process  when 
higher  order  dependency  is  measured  by  CorrfRt^ P) ,  ( Xt+r-p. )2 ]  or 
Corr[Rt(P),(Rt-t.r(P^  )2] ,  assuming  that  the  Rt(P)'s  are  uncorrelated . 


Theorem.   With  the  random  coefficient  model  (4.1), 

Corr[Rt(P),(Xt-r-n)2]  and  Corr[Rt(p>,  ( Rt-r(p*  )2  3  are  equal  to  zero  for 

r-1,2 

Proof:   Using  the  definitions  (2.5)  and  (4.1), 
r^P)  -  (Xt-n)-ai(Xt-i-M)-.  ..-ap(Xt-p-M) 

-  (At(1)-a1)Xt.1   +  (At(2)-a2)Xt-2  +  •••  +  ( At(p)-ap)Xt-p 

+  Bt-d-ai-  •  .-«p)M   •  (4.2) 

On  multiplying  (4.2)  by  (Rt_r(P))2  and  taking  expectations,  thus  obtaining 

Cov[R^(P), ( R^_r(P) )2] ,  we  have  a  sura  of  p  terms  given  by 

P 

E  E{(At(3)-aj)Xt_j(Rt_r(P))2}  (4.3) 

j-l 
and  a  last  term  involving  B^  which  is  clearly  zero.    Now  At^^-aj  is 
independent  of  both  the  X^-j  and  (Rt_r(p))2,  for  r-1,2,...,  which  may 
nevertheless  be  themselves  dependent.   Thus  the  jth  term  in  (4.3)  becomes 

E(ACn-aj)E{Xt-j(Rt-r(P)>2}  -  0        for  r-1,2 

since  E(A(J))«ctj.  This  completes  the  proof  which  clearly  includes  the 
first  cross  correlation  mentioned  in  the  theorem.  The  proof  highlights  the 
fact  that  it  is  the  independence  of  the  vector  of  coefficients 
(A^  '  i  •  •  •  ,  At' p '/  B^)  on  previous  X-^'s  uhich  creates  the  effect  in  this  type 
of  model . 

The  results  of  the  theorem  can  be  u3ed  to  help  validate  random  coef- 
ficient autoregressive  models;  also  useful  in  this  respect  are  the  non- 
zero higher  order  residual  crosscorrelations,  e.g.  for  positive  r 
Corr[Rt^p^, (Rt+r^P^ )2 3 •  These  have  been  obtained  for  the  second  order 
autoregressive  exponential  process  studied  in  Lawrance  and  Lewis  (1985); 
similar  results  for  any  first  order  random  coefficient  model  of  the  type 
(4.1)  are  given  in  the  next  subsection. 

It  is  worth  noting  that  with  G-t-r  defined  as  any  reasonable  function  of 

( Xt-r,Xt-r-i , . . .  ),  a  similar  argument  to  that  given  in  the  proof  shows  that 

Corr[Rt(P),Gt_r]*0,     r-1,2 (4.4) 

A  result  of  this  type  is  not,  however,   sufficient  to  establish,   for 
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instance,  that  R^P)  and  Rfc_r(P)  are  independent;  they  are  dependent. 

The  random  coefficient  autoregressive  structure  of  (4.1)  is  not  a 
necessary  condition  for  the  results  of  this  section  to  hold;  this  may  be 
illustrated  by  noting  that  CorrfR^P  >,  (X^_r-M)2  ]  is  zero  when 
C12(r)eE{(Xt-M)(Xt-r-n)2}  satisfy  the  equations 

Ci2(r)-a1C12(r-l)+a2c12<r-2)+- •  •+<xpCi2(r-p),  r-1,2 (4.5) 

The  similarity  of  these  equations  to  standard  Yule-Walker  equations  (2.4) 
will  be  apparent.  Thus  any  process  with  this  property  will  have 
Corr[Rt(P),(Xt_r-n)2]  equal  to  zero  for  non-negative  r. 

4.2  Higher  Order  Dependency  for  Pirst  Order  Random  Coefficient 
Autoregressive  Models 

The  model  to  be  considered  is  the  first  order  (p-1)  case  of  (4.1),  now 

to  be  denoted  as 

Xf-AtXt-j+Bt ,     t-0.  :1,±2,  .  .  .  ,  (4.6) 

in  which  A^  and  B^  are  independent  between  each  t,  but  within  themselves 
may  be  dependent;  we  also  write 

a-E(At),  n-E(  Xt )-(  1-a  )"1e(  Bt ) . 

Many  basic  mathematical  and  probabilistic  properties  of  this  equation  have 
been  studied  by  Vervaat  ( 1979 ) .  Interest  here  is  restricted  mainly  to  the 
residual  crosscovariances  of  (Rfc2,Rt-r)  and  [( X^-m  )2,Rt-rl  where  R<-  is  the 
first  order  residual  given  by 

Rt-Xt-M-a(  Xt_!-H ) .  (4.7) 

It  has  been  established  in  Section  4.1  that  Cov(  R^.  ,Rt_r2  )-Cov(  R^.2  ,  Rt+r  )*=0 
for  r-1,2,...,  and  so  now  Cov(Rt2,Rt_r )  for  r— 0,1,2,...  is  obtained 
explicitly. 

The  calculation  begins  by  writing  the  required  covariance  as 
Cov( Rt2 , Rt_r )-£{[(  Xt-M )-a(  Xt_x-M  )  ]2 [(  Xt_r-M )-a( X^j-^-n ) ] } 
-C21< r  )"aC2l( r+1 )-2a{C111(  r )-aC111(  r+1 )} 

+a2{C21(r-l)-aC21(r)}  (4.8) 

where,  as  defined  just  before  (4.5), 

C2i<  r  )-E{(  Xt-M  )2(  Xt_r-M  ) )  ,    Cm<  r  )-E((  XfH  )(  Xt-i-M  )(  Xt_r-n  ) )  . 
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The  higher  order  triple  moment  Cm(r)  is  easily  seen  to  be  equivalently 
given  by  aC2i(r-l),  in  the  present  instance,  and  hence  (4.8)  becomes, 

Cov(Rt2/Rt_r)-c21(r)-aC2i(r+l)-a2{C2i(r-l)-aC2i(r)},   r-1,  2 (4.9) 

For  r— 0,  with  ji-3"E[(  X^-y.  )3  ]  /  there  is  the  seperate  result 

E( Rt3 )-( l+2a3 )M3"3aC2i( 1 ) .  (4.10) 

The  calculation  of  C2^(r)  is  effected  by  writing  the  defining  equation 
(4.6)  in  the  form 

Xt-H-At<  Xt-1""M  )+(  MAt+Bt-M  )  .■ 
Squaring  both  sides,  multiplying  by  Xt-r~M»  ^Lnd  taking  expectations  then 

gives 

C21(  r  )-a2C21(  r-1  )+2(  ^a2+E(  AtBt  )-na  )C(  r-1 )  (4.11) 

where 

C(  r  )=Cov(  Xt ,  Xt-r  )-arcx2 ,    <723rvar(  Xt ) ,    a2=E(At2). 

Further  simplifications  of  (4.11)  give  the  recursive  equation 

C21(r)-a2C21(r-l)+kar_1,     r-1,2 (4.12) 

where 

k-2cr2  { Mvar(  At  )+Cov(  At ,  Bt ) } . 

Equation  (4.12)  has  explicit  solution 

C21( r  )-a2rM3+k( a2r-ar )/( a2-a ) ,  ( a2*a ) 

r-0,1 (4.13) 

-arM3+krar-1  ,  (a2-a) 

Going  back  to  (4.9)  for  Cov( R^2,Rt-r )  and  using  (4.12),  we  have  finally 

Cov( Rt2 , Rt-r )"(  a2+a3a2-aa22-a2 )C2^(  r-1 ) 

+(l-a2+a3-aa2)kar"~1,    r=>l,2 (4.14) 

Thus,   (4.14)   and   (4.13)   constitute   the   explicit   solution   for   the 

crosscovariances  of  the  residuals  (Rt-2,Rt_r);  notice  that  when  A^  is  the 

constant  a,  the  case  of  the  standard  first  order  linear  model,  these 

crosscovariances  are  correctly  zero.   The  NEAR( 1 )  model  of  Lawrance  and 

Lewis  (1981)  specializes  (4.6),  by  having  independence  within  each  (At,Bt) 

and  the  particular  forms 

At  "  f?  W,P'  ?   .       ^  -  KtEt,       Kt  -  P  W'P-  f 
^    L0  w.p.  l-a       ^  r    lb  w.p.  1-p 

where 

b  =  (l-a)0,   p  -  (  1-/3  )/{l-(  l-a  )/3} 

and   (Kt)   is   an   independently  distributed   exponential   sequence   with 
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parameter  X. 

If  the  covariances  (4.14)  are  to  be  converted  to  correlations,  then 

Var(Rt)  and  Var(R-t*)  must  be  obtained.    This  requires  the  following 
calculation, 

Var(Rt2)  -  E(Rt*)  -  [E(Rt2)]*  (4.15) 

where 

E(Rt2)  -  (1  -  a*)<r*  -  Var(Rt).  (4.16) 

E(Rt4)  -  (l+a*)M4  -  4aC31(l)  +  6a*C22(l)  -  4a3C13(l),         (4.17) 
with 

Ji4-EC<  Xt-M  )4]  ,  Ci j(  1  )-E[Xt-M  )i(  Xt_!-n  )  J  ] ,  (  i,  j  )-(  3  , 1 ) ,  (  2 ,  2  ) ,  (  1 ,  3  ) .  (  4. 18  ) 

The  joint  moments  in  (4.17)  are  now  obtained  in  terms  of  the  first  four 

moments  about  the  mean  of  Xt,  together  with  the  first  three  moments  and 

joint  moments  about  zero  of  At  and  B^.    Thus  in  terms  of  the  further 

quantities 

a3  -  E(At3),  b2  -  E(Bt*>,  dij  -  E(AJBJ),    (  i,  j  )-( 1, 1 ),  ( 1,  2  ),  (  2  , 1 )  (4.19) 

we  have,  as  with  the  derivation  of  (4.11),  that 

C31(l)  -  3M2o-*a-3K(M3+2M<7*)a2+(/i4+3wi3+3M^)a3 

-€M<J'2dii+3a*d12+3(^3+2Mffi)d21,  (4.20) 

C22(l)  -  -M2o-*-2MM3a+(  /i4+2MM3+^<y2  )a2 

-HT2b2+2(M3+MCT2)d11,  (4.21) 

C13(l)  -  n4a.  (4.22) 

The  results  ( 4. 20 )-( 4. 22 )  have  been  checked  against  an  alternative  method 
of  calculation  involving  moments  about  zero  instead  of  about  ji- 
lt was  remarked  in  Section  3,  following  (3.2),  that  the  cross- 
correlations  of  (X^-m)2  and  Rt+r^P)  can  also  be  useful  in  assessing  higher 
order  dependence.  In  the  present  case  of  first  order  residuals  and 
autoregression  there  are  the  results, 

Cov[(Xt-M)2,  Rt-r]  "  c2l(r)  "  a  C21(r+1),  (4.23) 

var(Rt)  -  (l-a2)<r2,  Var[(  Xt-M)2  )]  -  M4  -  <r*  •  (4.24) 

These  formula  can  all  be  applied  to  the  NEAR(  1 )  model  of  unit  mean  by- 
simp  ly  noting  that  in  this  ca3e 
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E(Atr)  -  a0r,    E(Btr)  -  rl[p+<l-p)br].  (4.25) 

Figures  1  and  2  give  the  results  of  computations  of  Corr[Rt2 ,Rt_r]  a-n^ 

CorrCCXt-M.)*. Rt-r]  for  tne  NEAR(l)  model.  The  top  left  frame  of  Figure  1  is 

pn         srfi-  .  ■.  sum 

the  linear  EAR( 1 )  ca3e  for  which  all  cross-correlations  apart  from  lag  zero 

are   zero;   the   other   three   cases   in   Figure   1   each   have   zero 

crosscorrelations  at  negative  lags,   in  agreement  with  the  theoretical 

results  in  Section  4.1,  but  have  some  non— zero  values  at  the  zero  and 

positive  lags.   It  is  evident  that  the  lag  zero  cross  correlations  contain 
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Figure  1.  Four  computations  of  the  crosscorrelations,  for  various  lags, 
between  the  linear  autoregressive  residual ,  R$(P)  and  (Rt^^  )z  f°r  the 
NEAR/ 1 )  process  uith  p(i)=a0  held  constant  at  0.75;  in  effect  the  remaining 
free   parameter    is   being   varied    through    its   allouable   range. 
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much  of  the  discriminating  information  between  the  four  cases;  this  will  be 
so  for  first  order  autoregressive  models  in  which  much  of  the  higher  order 
dependency  is  at  lag  one,  and  the  lag  zero  cross  correlation  involves  both 
third  and  fourth  order  aspects  of  the  lag  one  dependency.  For  further 
information  on  the  TEAR(  1 )  and  PREAR(  1 )  cases  see  Lawrance  and  Lewis 
(  1981). 

Figure  2  gives  Corr[(  X^-p.  )z  #R-t-r  1  f°r  ^e  same  four  cases  used  in 
Figure  1.  The  negative  lags  are  again  zero  for  all  cases,  and  hence  this 
property  does  not  discriminate  the  linear  from  the  nonlinear  cases.  Rather 
it  is  the  strength  of  the  crosscorrelations  at  positive  lags  which  performs 
this  task,  albeit  less  clearly  than  the  crosscorrelations  of  the  residuals 
and  squared  residuals. 
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Figure  2.  Four  computations  of  the  crosscorrelations ,  for  various  lags, 
betueen  the  linear  autoregressive  residual ,  Rt(P),  and  (Xt-u)z  for  the 
NEAR(  1 )  process  uith  p(l)-=a&  held  constant  at  0.75;  in  effect  the  remaining 
free   parameter    is    being    varied   through    its   allouable   range. 


15 


4.3   Higher  Order  Dependency  for  the  Product  Auto  regress  ion  Model 

Another  form  of  nonlinear  autoregressive  model,  called  PAR(l),  was 

introduced  by  McKenzie  (1982);  the  basic  idea  of  its  construction  is  to 

consider  the  exponentiation  of  the  standard  AR( 1 )  equation.    Then  its 

additive  structure  becomes  multiplicative,  and  the  general  form  of  the 

PAR(  1 )  product  autoregressive  model  model  is 

Xt-CXt-xJPBt,    t-0,±l,±2,...,  (4.26) 

where  p  is  the  dependency  parameter,  0<p<l,  and  {B^}  is  an  independent  and 

identically  distributed  innovation  sequence,  independent  of  Xt-i,Xt_2, . . .  • 

McKenzie  studies  the  model  when  B^  is  chosen  so  that  (X^.}  has  a  gamma 

marginal  distribution,  and  obtains  several  basic  results;  for  instance, 

that  p  still  represents   the  lag  one  autocorrelation  and  that   the 

autocorrelations  in  general  satisfy  the  Yule-Walker  first  order  equations 

((2.4)  with  p-1),  so  that  p(r)*pr.    It  is  apparent,  however,  that  the 

linear  conditional  expectation  definition  of  autoregression  (equation 

(2.4))  is  not  satisfied  because  of  the  power  form  implied  by  (4.26). 

As  with  the  first  order  random  coefficient  models  in  Section  4.2,  we 

consider  the  first  order  residual  R^,  given  by  (4.7),  and  will  likewise 

determine  Cov(  R^2,R^_r )  for  r=0, ±1, *2, . .  . ,  noting  that  for  this  model  these 

correlations  are  non-zero  for  all    lags.      We  will  u3e  the  general  expression 

(4.8)  in  terms  of  the  third  order  central  moments  C2^(r)  and  Cm(r),  but 

this  time  there  is  no  simple  relation  between  them,  and  both  are  needed  for 

all  lags.   Also,  calculation  of  C2i(r)  and  Cm(r)  must  be  in  terms  of 

their  uncentered  components,  since  these  are  the  quantities  which  can 

immediately  be  determined  from  the  PAR(  1 )  equation  (4.26).   The  required 

uncentered  moments  will  be  written  as 

e2=E(Xt2),    e3=E(Xt3) 

(4.27) 
e2i(  r  )-E(  Xt2Xt_r  ) ,    exll(  r  )-E(  XtXt-xXt-j- ) 

for  r=0, ±i, ±2, . . . ,  and  there  is  need  to  note  the  special  cases, 

e21(0)se3,    e111(0)se2l(l)'    elxl(  1  )-e21( -1 ). 
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All  these  quantities  will  be  calculated. 

First,  it  is  necessary  to  note  the  following  relations  between  the 
centred  and  uncentred  moments, 

C2i(r)-e2l(D-M3-2MC(r)-M<72        (all  r) 

Clll<0)-C2lU>'    Cm(lK2l(-l)  (4.28) 

Clll<  r  )-eiu(  r  )-m3-ji(C(  r  )-KT(  r-1  )+C(  1 ) } ,   (  r*0,  1 ) . 

The  calculations  of  e2i(r)  and  em(r)  need  to  be  treated  separately 

for  positive  and  negative  lags,  but  follow  in  the  same  general  manner  and 

will  be  illustrated  by  that  for  e2i(r)  for  positive  lags.   By  iterating  the 

PAR( 1 )  equation  (4.26)  r  steps  backward, 

r-1 
Xt  -  Xt_rP<r)  n  Bt-i^i) 
i-o 

where  p( r )»pr, r«0, 1, . . . .   Squaring  this  equation,  multiplying  it  by  X^-r 

and  taking  expectations,  gives 

r-1 
e2i(r)-E{Xt_r2P<r)+1)  H  EfBt-i2^1)}.  (4.29) 

i-0 

To  obtain  the  expectations  in  the  repeated  product,  taking  the  2p( i  )th 
power  of  (4.26),  leads  to 

E(Xt2P<  *  >  }-E{Xt-i2p*<  i  )  }E{Bt2P<  *•  > } , 
and  hence,  dropping  the  unnecessary  suffix  t,  to  the  result 

E{B2p(  i ) }-E(x2P(  i ) }/E{X2P<  i+1 > } . 
Now  (4.29)  can  be  expressed  purely  in  moments  of  X,  as 

e2i(  r )-E{X2P<  r  >+1)E( X2 )/E{X2P(  r > } .  ( 4. 30  ) 

To  proceed  further,  invoke  a  gamma  marginal  distribution  for  X  with  density 

f ( x )-e0x0-1e_ex/r( & ) ,   e,£>o,  x>o;  (4.31) 

this  has  mean  (3/9,  variance  <3/©2 ,  third  central  moment  2/3/e3,  and  there  is 
the  kth  moment  result  E(  Xk  )-r(  /3+k  )/©kr(  p ) .  From  (4.30)  and  similar 
expressions  we  then  have 

,      f/3(0+l)(0+2pr)/e3,  r-1, 2 ,432) 

*2l(r>  -  [^(0+plrl  )(p+plr|+1)/e3/      r— 1,-2,...        (     ' 

,     >    f/3(0+p)(/3+pr_1+Pr)/e3/        r-2,3,...  u„, 

6lll(r)  "  U(0+p'r|)(0+P+P,rl+1)/e3,   r~l,-2,...       (4l33) 
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with  e2i(0),  em(0)  and  em(l)  being  given  by  the  special  cases  of 

(4.27).   Use  of  (4.28)  gives  finally  the  required  expressions 

[2/3pr/©3  r-0,1,...  ,      ... 

C21(r)   '    Uplrl+P2|rl)/e3,  r~l,-2,...  (4-34) 

r        (vs  .      f/3(Pr+Pr+1)/©3/  r-2,3,...  /aocx 

Cm(r)   -    (^{p|rl+l+p2|r|+l)/e3  r— 1,-2,...  (4'35) 

All     these     may     be     used      in     (4.8)     to     obtain     the     desired     result      for 

Cov(  R^  2  ,  Rt-r )   as 

f20(l-p2)2pr/e3                                 r-1,2,... 
Cov(Rt2,Rt_r)    -      ,3(2-6p2+p3+3p4  )/©3                            r=0  (4.36) 

[-/3(l-p)(l-pi)ip2'rl-1/©3  r— 1,-2 

The    simpler    covariance    of    (X^-m)2    and    Rt-r    also     follows    via    (4.23)    and 

(4.35  )   as 

Conversion  of  these  covariances  to  correlations  requires  Var(Rt2)  as  at 
(4.15)  and  Var[(Xt~K)2]  as  at  (4.24);  the  required  intermediate  results  are 

M4  «  3/3</3+2)/e*, 
C31(l)  -  30(0+2)p/9*,  C13(l)  -  0{3/3+(l+p)(2+p)}p/e*, 

C22(l)  -  {/32+20p+20(0+2)p2}/e*.  (4.38) 

These  then  give  as  at  ( 4. 15 )-( 4. 18  )  the  explicit  expression 

Var(Rt2)  -  20{(0+3-2(/3+6)p2+6ps+(/3+ll)p4-6ps-2p6}/e*.  (4.39) 
Together  with  Var(  R^- )  which  is  simply  (l-p2)/3/©2,  Cov(  R^.2  ,Rt_r )  can  be 
converted  into  Corr( Rt2,Rt-r ) •   Similarly,  (4.37)  requires  the  result 

Var{(Xt-M)2}  -  20(/3+3)/9*  (4.40) 

in  order  to  obtain  corr{(Xt-ji)2  ,Rt }  • 

Figures  3  and  4  illustrate  the  residual  and  squared  residual  cross 
correlations  for  the  PAR( 1 )  model.  Figure  3  shows  how  the3e  residual  cross 
correlations  vary  over  the  range  of  p  values  from  p=0  to  p=0 . 9  for  an 
exponential  marginal  distribution.  At  p=-0,  the  PAR(  1 )  is  an  IDD  process 
and  the  residuals  are  trivially  independent,  resulting  in  the  only  non-zero 
crosscorrelation  at  lag  zero.  As  the  p  value  increases  the  dependence 
spreads  out,  mosc  strongly  at  positive  lags.   An  interesting  feature  is  the 
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lag  zero  crosscorrelation  which  changes  from  being  strongly  positive  at  p-0 
to  approximately  zero  at  p— 0.75,  to  moderately  negative  at  p-0 . 9 
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Figure  3.  Computations  of  the  residual  crosscorrelations ,  Corr(Riz,R*.r) 
for  the  PAR(l)  model.  Since  /3-i  the  marginal  distribution  is  exponential. 
The  lag  one  serial  correlation  is  increased  from  p(l)=0.0  (upper  left)  to 
p( 1)^0.9    (louer   right). 

Figure  4  gives  four  different  gamma  cases  of  the  PAR( 1 )  residual  cross 
correlations,  all  with  p-0. 75.  The  gamma  shape  parameter  0  takes  the 
values  0.5,  1.0,  2.5  and  10.0;  these  cases  indicate  that  changes  in  the 
gamma  parameter  cause  only  slow  changes  in  detail  of  the  cross  correlations 
which  from  the  formulae  (4.37),  (4.39)  and  (4.40)  all  tend  to  zero  as  0-». 
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Figure  4.  Computations  of  the  residual  crosscorrelations ,  Corr(Rtz ,Rt.r) , 
for  the  PAR(l)  model.  The  p  for  each  case  of  0.75,  and  the  figures 
illustrate  the  effect  on  the  residual  crosscorrelations  of  changing  the 
index  Q   of   the   gamma   distribution   through    0.5,    1.0,    2.5   and   10.0. 


5 .   CONCLUSIONS 

A  methodology  for  analyzing  higher  order  dependence  in  nonlinear  time 

series  with  pth  order  autoregressive  correlation  structure  has  been 

proposed.     It   utilizes   standard   uncorrelated   linear   autoregressive 

residuals,  and  the  crosscorrelation  function  of  these  residuals  and  their 

squares.   The  behaviour  of  this  crosscorrelation  function  has  been  utilized 

for  two  rather  different  types  of  nonlinear  model:   random  coefficient 

autoregression  and  multiplicative  autoregression:  the  behaviour  has  been 
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shown  to  allow  discrimination  between  models  in  the  same  class  with  the 
same  marginal  and  autocorrelation  structures. 

The  residuals  crosscorrelation  function  provides  a  partial  analysis  of 
third  order  information  in  the  time  series;  it  does  not  attempt  to  capture 
all  third  order  information,  which  is  the  aim  of  such  techniques  as 
bispectral  analysis  and  which  will  often  be  intractable  with  nonlinear 
models.  Being  based  on  standard  linear  residuals,  the  analysis  extends 
rather  than  replaces  conventional  residual  analysis. 

Developments  of  the  analysis  which  focus  on  the  directionality  implicit 
in  many  time  series  are  being  investigated;  reversed  residuals  assume  a 
reversed  directionality  and  allow  exploration  of  the  consequences  of  such 
an  a3S3uraption. 
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